SMOOTH STRUCTURE ON THE MODULI SPACE OF INSTANTONS OF A 

GENERIC VECTOR FIELD 



DAN BURGHELEA 



Abstract. This paper is a sliort version of some joint work with Stefan Haller. It de- 
scribes the structure of "smooth manifold with comers" on the space of possibly broken 
instantons of a generic smooth vector field. The result is stated in Theorem 1.4. 
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1. The results 

Let M be a smooth closed manifold and X : M ^ TM a smooth vector field, i.e. a 
section X : M TM in the tangent bundle. The set X{X) of rest points of X consists 
of points of M where the vector field vanishes, X{X) {x E AI\X{x) = 0}. 

For any x G X{X) the differential DX of the smooth map X defines the endomorphism 
D^{X) : T^{M) T^{M) called the linearization of X at x[] 

The rest point x e '^{X) is called hyperbolic if the eigenvalues of Dj.{X)^ {A G 
Si^tclD x{Xy} , are complex numbers with real part 5RA ^ 0. In particular Dx{X) is in- 
vertible. 

The hyperbolic rest point x S X{X) is called of Morse type if one can find coordinates 
{ui , M2 , • • • , u„) in the neighborhood of x such that X ~ ±Ui . 

Given a hyperbolic rest point x € X{X) the cardinality of the set of eigenvalues counted 
with multiplicity whose real part is positive is called Morse Index and is denoted by 

ind(a;), 

ind(2;) := tt{A G SpectD^(X)|3fiA > 0}. 



1991 Mathematics Subject Classification. 57R20, 58J52. 
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^Dx {X) is defined as follows. Choose an open neighborhood U of x in M and a trivialization of the tangent 
bundle above U,e -.TU ^ U X Tx{M), with 0\t.^{m) = id. Consider Y := pr^ -0 ■ X : U ^ Tx(M) with 
pr2 the projection on the second component; Y(x) = 0. Observe that D^iY) is independent of d and defines 
D^{X) := D^{Y). 
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A trajectory of X is a smooth path 7 : M — > A/ so that ^ ~ X{'j{t))Q One denotes 
by 7y, ?/ e M, the unique trajectory which satisfies 7^(0) = y. 

A trajectory 7 is called instanton from xtoy,x and y rest points, if limf^_oo j{t) = x 
and limf^oo j{t) = y. 

lix £ X{X) the set 

lim 7^=4 
^±00 

is called the stable / unstable set of x. 

Note that any point x E M lies on a trajectory but not necessary on an instanton. For 
x,y G X{X) the set of points lying on instantons from x to y is denoted by M{x, y) and 
is exactly the intersection 

M(x,y) ■.= W-r\W+. 
The additive group of real numbers M acts on M{x, y) by "translation" ; if the action is 
denoted by /i : ]Rx7W(x,i/) M{x,y),a € R,j G M{x,y),\hen ij,{a,j){t) = 7(t+a). 
The quotient set of this action, 

T{x,y) ■.^M{x,y)/R, 

is actually the set of instantons from x to y. 

A smooth function / ; i\/ M is called Lyapunov for X if X„i{f) < 0, for any 
me M\XiX). 

A smooth closed one-form uj G ri^(Af) is called Lyapunov if Ld{X)„i < for any 

X e M\x{x). 

Not any vector field admits Lyapunov closed one-forms and a vector field can have a 
Lyapunov closed one-forms but not Lyapunov functions. If X has Lyapunov functions 
then any trajectory is an instanton]^ and there are no closed trajectories. While Lyapunov 
function might not exist, for any hyperbolic rest point x G -^(X) there exist open neigh- 
borhoods U of X and smooth functions f : U ^M. Lyapunov for X\u. Similarly, for any 
instanton 7 G T{x,y), when x,y are hyperbolic rest points, there exist open neighbor- 
hoods [/ of 7 and smooth functions f : U ^ M. Lyapunov for X\if. 

The first important result about stable/unstable sets is the following theorem due to 
Perron and Hadamard, cf. ifTTl Theorem 17.4.3, [11 or Q Theorem 6.17 . 

Theorem 1.1. If x is a hyperbolic rest point then resp. is the image of a one to 
one immersion xt ■ E""'"'^^ -> M resp. Xx ■ R™^"^ M. 

In fact one can find immersions Xx '■ E"~'"'^^ x R™'^^ M with Xx(0) = x, 
Xx|(R"-i"d^xo) ^ xt and Xx|(oxRi"d^) ^ X.x ■ 

If X admits a Lyapunov function then xt resp. Xx is actually a smooth embedding 
which makes resp. a smooth submanifold of M. In general the topology of 
resp. W~ obtained by identification with ^"-"1^2; j-ggp ]ginda;^ referred below as 
manifold topology, is not necessary the same as the induced topology |^ The immersion 
xt is not unique but the manifold topology on and the smooth structure defined by the 
chart xt is- It is possible that for a hyperbolic rest point x the stable set and the unstable 
set be the samerl but the manifold topologies are different. 



If M is not compact then R should be replaced by an open interval, the maximal domain of the trajectory; 
when M is compact this domain is R. 
■'xhis is not true if M is not closed. 

The later being coarser in general. 
^In this case dim M has to be even. 



SMOOTH STRUCTURE ON THE MODULI SR\CE OF INSTANTONS OF A GENERIC VECTOR FIELD 



3 



In conclusion is equipped with a canonical structure of smooth manifold such that 
the canonical inclusion : Af is a one to one immersion (embedding if X admits 

Lyapunov functions). Suppose x,y E are hyperbolic and the maps i~ and i+ are 

transversal. Then the set 



has a structure of a smooth manifold of dimension indx — indy with the canonical in- 
clusion ix^y : M{x,y) — > Af a one to one smooth immersion and the action /i : E x 
M{x,y) — >■ M{x,y) smooth and free provided x ^ y. In this case the quotient space 
T{x, y) receives a canonical structure of smooth manifold of dimension ind x — ind y — 1 
with the quotient map p : A4{x,y) — > T{x, y) a smooth bundle. Clearly if ind x < ind y 
and X y then A4{x, y) is empty. 

From now on we suppose the vector fields satisfy the following two properties : 
Pi : All rest points of X are hyperbolic. 

P2 : For any two rest points .t, y E X{X) the maps and z+ are transversal. 

The following result due to Kupka and Smale, cf (HI, ifTOl . ||9l . shows that this is the 
generic situation. 

Theorem 1.2. The set of vector fields which satisfy Pi and P2 are residual^in the — 
topology for any r > 1. 

Write X > y for ind x > ind y when x,y E X{X). 
For any fc > 1 , introduce 



Mix,y) e W- x W+ \i- {u) = t+ (v)} 



W-{k) : 



u 



Tiyo,yi) X 



T{yk-i,yk) X 



Vk ' 



ya > yi > ■ ■ ■ > Vk 
X - yo 



W+{k) : 



u 



T{yi,yn) x 



nyk,yk-i) X W, 



r + 
Vk ' 



yo < yi < ■ ■ ■ < yk 



X — 



yo 






yo > yi > y2 > ■ ■ ■ > yk > Vk+i 
^ = yo. !/fc+i = y 




ya > yi > ■ ■ ■ > yi 
yi+i > yi+2 > ■ • ■ Bfc+i 
^ = vo,Vk+i = y 



and define the maps ij{k) : W!^{k) — > M and ix,y{k) ■ M{x, y){k) — > M by 




yo < yi < ■ ■ ■ < yk 
a: = yo 





I. 



opVM 



ya > yi > ■ ■ ■ > yt+i 
^ = yo. «fc+i = y 



i.e. contain a countable intersection of open dense sets 
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For k ~ write 

W^iO):^W-, T{x,y)iO):^T{x,y), Mix,y){0) -.^ Mix,y) 

and 

~ *x ' ^x,yiO) = ix.y 

Denote by , Tix, y) and M.{x, y) the sets defined by 



n>0 



"Tla^.y) := y T{x,y){n), 

n>0 



n>0 

and denote by : W"" — ?> M and i^^.^ : 7Vl(a;, y) — )^ M the maps defined by 

*J IvVjF (n) *J ('^)' 
^a:,y |T(a;,y)(n) ■ ^x.yi'^)- 

We equip W"^, T(x, y) and A^(a:, j/) with the transversal slice topology defined below. 
Since for the vector field X is the same as for the vector field —X and since 

M{x,y) = {Kt;) e M^- X W^/|(z-)(u) = t+iv)}, 

it suffices to describe only the transversal slice topology for and T{x, y). A few more 
definitions are necessary. 

A broken instanton 7 = (7^, 7^, • • • 7*^) consists of a collection of instantons 7' 's 
with the property 

lim 7'(t) = lim 7'+\t), i = 1, 2 • • • , fc - 1. 

i— >-oo i— > — 00 

An element of T(x, y) is a broken instanton with the property 

lim 7^(t) = X, lim 7''(t) = y. 

An element of can be uniquely represented as a pair 7 := (7,771), with 7 = 
(7^, 7^, • • • 7*^) a broken instanton and m E M which satisfy: 

1. limt^_oo 7HO = 2;, 

2. limt^oo 7''(i) = limt^_oo 7m(i)- 

Given a collection of trajectories 7 = (7^, 7^, • • • 7*^) with the property limt_j.oo 7^{t) = 
limt_>_oo 7*^^(^) (i.e. a broken trajectory) a transversal slice is a collection U = 
{Ui, U2, - ■ ■ Ur) of disjoint {n — 1) dimensional submanifolds diffeomorphic to open discs 
which are transversal to X (transversal to trajectories of X) and satisfy: 

1 . each ji intersects at least one Uj , 

2. consecutive [/js intersect either the same or consecutive 7*'s, cf FIGURE[T| As a 
consequence we have r > k. The intersection points of the submanifolds [/js and trajecto- 
ries 7* 's are called marking points. 

For a system (7, V, U_) with V open neighborhood of 7 in M and U_ a transversal slice 
to 7 denote by V(7, V^, U_) the collection of possibly broken instantons in Tix, y) which 
lie inside V and have i7 as a transversal slice. 
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Figure 1. 

For a system (7, V, U_, O) with 7 e , V open neighborhood of 7 in M IJ_ transver- 
sal slice for (71, • • • 7r, 7m) and O an open neighborhood of rn, see FIGURE [1] denote 
by V(7, V, IZ, O) the collection of all elements in which lie inside V, have t/ as a 
transversal slice and have the end point in O. 

A base of the transversal slice topology for T(a;, y) resp. for is provided by 
the sets V(7, V, C/) for all systems (7, V, C/) resp. the sets V(7, V,IJ_, O) for all systems 

(7, v,u, O). 

Theorem 1.3. 1. When equipped with the "transversal slice" topology the sets ,T{xTy) 
and Ai{x,z) are Haussdorf paracompact spaces and the maps i~ : W~ — M and 
ix,z '■ M.{x,z) ^ M are continuous. 

2. If X admits Lyapunov function then , T{x, y) and Ai{x, z) are compact. 

Proof, (sketch). Suppose first that X admits a Lyapunov function f : M ^ M. 

The case ofT{x,y): Suppose f{x) — ci and /(y) = C2 and choose < Ci < 
ai < a2 < • • • afc^i < C2 < at with a; regular values and with the intervals (a^, Ofi+i) 
containing only one critical value c^+i . The map which assigns to a broken instanton 7 its 
intersection with the levels /~^(Q;i)'s, 

i:f{x,y)^ \{ f-\a,), 

is a one to one map. It is not hard to see that the transversal slice topology is the same as 
the topology induced by this embedding. Indeed one can consider only transversals which 
lie in the levels f~^{ai) and show they suffice to describe the transversal slice topology. 
Standard arguments cf. [3 1 show that the image of i is closed hence compact. This proves 
the result for T{x,y). 



6 



DAN BURGHELEA 



The case of : Write the critical value s in decreasing order ■ ■ ■ > Ci > Ci+i > 



Verify the first assertion in Theorem 1.3 for (i^ ) ^(/ ^ (a, 6)) instead of W'^ when 



a, b are regular values of / with at most one critical value in the interval (a, b). The set 
(i~)~^(/~^(a, b)) can be embedded in a product of finitely many levels of / and M and 
one can check that the transversal slice topology and the topology induced by such em- 
bedding are the same. Since i~ is continuous w.r. to the transversal slice topology, hence 
{i^)^^{f^^{a, b)) is open and W~ is covered by such sets, then the conclusion extends 
from (i^)^^(/~^(a, b)) to W~ . The compacity assertion follows from the compacity of 

To conclude the statement in general (when no Lyapunov function exists) one observes 
that any 7 G T{x, y), or (7, m) e lies inside an open set V of M so that the vector 
field X\v admits Lyapunov function f : V ^ R. This follows from the existence of 
Lyapunov functions in the neighborhood of each hyperbolic rest point a fact noticed above. 
More details will be contained in |5 1. 

The main result of this paper states that the topological spaces , T{x,y) and 
A4{x,y) have structures of smooth manifold with corners. To explain this let us recall 
a few definitions. 

The standard example and the local model of a smooth manifold with corners is 

The fc— corner of M" is 

9feM" := {x e M" [exactly k coordinates Xi = 0}. 

Denote by [[n]] the set {1, 2, • • • ,n — 1, n}. If I = {ii, 22, • • • ik} is a subset of [[n]] 
denote by the set of points in M" whose coordinates ^ , Xj, , • • • are different from 
zero while all other coordinates vanish. Note that each carries a canonical orientation 
defined by the order ii < Z2, • ' ' < ife- 

A smooth manifold with corners is a Haussdorf paracompact space P equipped with 
a differential structure locally isomorphic to M" . A differential structure is given by an 
equivalence class of atlases. An atlas {(^^ : C/q, — > Vq, a e A} consists of 

an index set A , 

C/q C P open sets of P, 

Va C Mf: open sets ofR^, and 

ipa homeomorphisms (charts) 
so that UqVq = P and ipi^ ■ (p~-^ are smooth and of maximal rank where defined. Two 
atlases are equivalent if their data considered together remain an atlas. 

The /c— corner dkP is the set of points which in some chart (and then in any) correspond 
todkR^. 

The manifold with corners is orientable if SqP is orientable. An orientation on such 
manifold is an orientation for its tangent bundle, equivalently an orientation of the open 
manifold OqP. 

A smooth manifold with corners P is clean if the closure of each connected component 
of the comers is a smooth manifold with corners. 

The main result of this paper is the following: 

Theorem 1.4. Let X be a smooth vector field satisfying Pi and P2 ( defined before Theorem 
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1. There exists a canonical structure of clean smooth manifold with corners on , 
Tixj-y) and A4{x,y) with 

dkW- = W^{k), dkf{x,y)^T{x.,y)ik), dkM{x,y) ^ M{x,y){k) 

and so that i^^y are smooth maps. 

2. If the rest points of X are of Morse type there exists an additional structure of smooth 
manifold with corners (different but diffeomorphic to the structure stated in 1.) with the 
same corners and the identity map restricted to each k^corner a diffeomorphism. 

3. Both T and Ai are equipped with (stable) framings. A collection of orientations 
{ox, X e X{X)} on induces coherent orientations on jCi[x, y) and T(a;, y)^ 

Theorem 1 1.4| is not new but in the generahty formulated above inexistent in literature. 
In less generahty it can be recovered from [6 1 and 1 1] for the gradient of a Morse function 
and from 12], f3l, H or ||6l for the gradient of a closed one form. The proof below is along 
the lines of |2| or [|3]. In 13 the result will be proven for a more general class of vector 
fields, called HB (hyperbolic - Bott) vector fields. For these vector fields the set of rest 
points X{X) is a smooth submanifold with hyperbolic in normal directions of X{X). 

The smooth structure provided by Theorem ] 1 .4| is not the only possible canonical smooth 
structure. In fact, in the case the rest points are of Morse type. Theorem 1 1 .4| provides two 
such canonical structures never the same. However all smooth structures of manifold with 
corners on W^f , M{x, y) or T(a;, y) which extend the smooth structure of , M{x, y) 
or T{x,y) are diffeomorphic. By elementary smoothing theory one can show that such 
diffeomorphisms can be chosen to be the identity on arbitrary closed subsets of the do part. 

Theorem [L4] provides a source of new invariants which deserve attention and we plan 
to explore in future work. For example: 

1 . A Morse type complex can be assigned to a class of vector fields substantially larger 
than the gradient like vector fields. Its homology/cohomology referred to as the instanton 
homology (cohomology) might relate the topology of the manifold and the dynamics of X 
in a more subtle way than in the case of gradient like vector fields. For example if X is the 
gradient of a closed one form both the Novikov cohomology and the cohomology of M 
twisted by a closed one form can be obtained as instanton homology/cohomology. More 
general vector fields lead to more subtle instanton homologies / cohomologies. 

2. A chain/ cochain complex can be derived from the corners structure of the mani- 
fold with corners T{x, y). The homology/cohomology of such a complex, referred to as 
the incidence cohomology seems natural to investigate. It carries significant dynamical 
information not obviously related to the topology of the manifold. 

3. The stable framing of T{x, y) can be used to define elements in 7rf (ilA/), the stable 
homotopy groups of the free loop space of M. A parametrized version of such elements 
might provide a more analytic understanding of the relationship between the homotopy of 
the space of diffeomorphisms and the Waldhaussen K- theory of the underlying manifold. 

Acknowledgement: I thank Stefan Haller for pointing out a number of errors and mis- 
prints in a previous version of this manuscript. 

2. Some basic ODE 

Recall that a linear transformation of M" is hyperbolic if all its eigenvalues have non 
vanishing real part. The stable resp. unstable subspace, M+ C M" resp. Mr C M", are 



This means that for any three rest points x,y,z with indx > indj/ > indz the orientation Ox,z on 
T{x, z) induces on T{x, y) X T(y, z) C &T{x, z) the oposite of the orientation Oa^^y ® Oy^z . 
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the sum of generalized eigenspaces corresponding to the eigenvalues with negative resp. 
positive real part. 
Consider 

(a) A e M{n x n; M) hyperbolic with stable space M+ = M'^ x and unstable space 

= X R"--'', 

(b) g : M" ^ M" = M+ X a smooth map with compact support with f;(0) — 
and L)o5 = 0. We write g = {9^,9^)- We require in addition that g^{x~^, 0) — 
0,5-(0,x-) = 0. 

Let X : M" ^ M" be the smooth map defined by X{x) = Ax + g{x). Regard X as 
a smooth vector field on M". Clearly G M" is a hyperbolic rest point and the only rest 
point in a small neighborhood of 0. 

Denote by 

j{t:p,q,Ti,T2) (7+(t : P, 9, Ti, T2), 7" (i : p, Ti, T2)) G M+ X 
{p, q) e M+ X , Ti < r2 a trajectory which satisfies 



l^iTi : P:q,Ti,T2) = p, 

1~{T2 ■.p,q,Ti,T2) = q. 

In general such trajectory might not exist and even if exists it might not be unique. 
However we have : 

Theorem 2.1. For any positive integer N there exists e, p,C > Q so that : 

1. For any {p, q) G (e) x (e), the discs of radius e in M+ x , and any Ti < T2 
there exists a unique trajectory ^{t : p, (7, Ti, T2). 

2. Moreover the following estimates hold 

||T7+(t:p,<z,Ti,r2)||<eC-''(*-^^) 
||T7-(t :p,<z,Ti,r2)|| <eC''(*-^^) 

/orTi < i < T2, whereT= with ki + k2 + fcg + + < N 

The result is a straightforward application of contraction principle. A proof can be 
derived on the lines of the proof of Theorem A. 2 and Lemma A. 3 of Appendix of HI. For 
the reader's convenience we sketch the proof of (L) and comment on the proof of (2.). 

We continue to write p and g+ for {p,Q), and (g+,0), and q and for (0, g) and 
(0,g~). We also write ^{s,t) fore^*"*)^. 

Using the hyperbolic linear transformation A one can produce the real numbers p' > 
and C > 1 so that 

\\-f(s, t)p\ I < Ce~p'^'-''> I \p\ I for s > t, 
< CeP fors < t. 

Since the trajectory ^[t) :— j{t,p, q, Ti, T2) has to satisfy the equaUty 

7W = ^{t,Ti){p) + ^{t,T2){q)+ [ ^{s,t)g+{^{s))ds- f ' ^{s,t)g- {j{s))ds (4) 

JTi Jt 
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one concludes that 7(i) is a fixed point of the map F : C°([Ti, Tz], M") -> C°([ri, T2], M") 
^defined by 



+ 1 



F{x{t)) - ^{t,T^){p)+^{t,T2){q)+ / ^{s,t)g+{x{s))ds- / ^{s,t){^,g- {x{s))ds. 

JTi Jt 

(5) 

Choose i? > so that 

\\Dgix)\\ < B\\x\\. 
Then if 1 1 (p, g) 1 1 < e and | \x{t) 1 1 < ?/ one has 

\\Fix{t)\\<2Ce+'^^^. (6) 

One can find 77 and e' small so that F sends the disc of radius 77 into itself provided 
1 1 (p, 1 1 < e'- Precisely one chooses 77 to satisfy 

77 < 



4BC" 
and e' to satisfy 

These choices make both terms of the right side of the above inequality (j6]l smaller than 
77/2. Then \ \x(t)\\ < 77 implies ||F(a;(t))|| < 77. 

Note that the estimates remain true for any t]' < t] (when e' is appropriately chosen). 
Since we have 
BF 

II — II < CBri/2p'{l - e" (^i^*)) + CB-q/2p' < CBr^lp' < 1/4 (7) 
ox 

by choosing 77 < p' /ACB and e' < 77/4C one concludes that F sends the disc ©(77) of 
radius 77 (in the complete metric space C°([Ti, T2], E")) into itself and that F : ©(77) — > 
0(77) is a contraction. 

To prove (2.) we first produce e, p with e < e' , p < p' so that (2) are satisfied for 
iV = 0, then decrease e and p inductively with N to satisfy (2.) 

This is done by incorporating the estimates stated in (2.) in the definition of the metric 
space C°([Ti, r2], M"), and decrease e, p to make sure that F remains a contraction even 
in the presence of these estimates, hence the unique fixed point 7 satisfies the estimates. 

3. Elementary differential topology of smooth manifolds with corners 

If P is a smooth manifold with corners then: 
9fcP is a a smooth (n — fc)— manifold, 
dP := |Jfc>i ^kP is a topological [n — 1)— manifold, and 
(P, dP) is a smoothable topological manifold with boundary. 

If Pi, i = 1,2 are two smooth manifolds with corners then the product Pi x P2 is a 
smooth manifold with comers with 

dk{Pi X P2) = □ 9fc'Pi X 5fe"P2. 

r—k' + k" 

If both Pi and P2 are clean then so is the product. 



^ C°([Ti,T2],R") denotes the Banach space of continuous function from [Ti,T2] to R" with the C°- 
norm. 
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Let P be a smooth manifold with comers, M a smooth manifold, 5 C M a smooth 
submanifold and f : P ^ M a smooth map. 

Definition 3.1. The map / is transversal to S, written / rh 5, if fld^p rh S for any 
fc = 0, 1, • • • n. 

Let Pi, i = 1, 2 be two smooth manifolds with corners, M a smooth manifold and 
fi'.Pi^M smooth maps. 

Definition 3.2. The maps /, : Pj ^ M are transversal, written /i rh /2, if the product 

fiX f2 ■■ Pix P2 M X M is transversal to the diagonal Am := {(a;, a;) e M x M\x e 



The above definition can be extended to a finite set of smooth maps fi'.Pi^M from 
manifolds with comers Pj to a smooth manifold M. 

Theorem 3.3. 1. If f &\ S then f~^{S) is a smooth manifold with corners (smooth sub- 
manifold of P) with dkf~^{S) = (/|afcp)~^(5'). Moreover if P is clean then so is f~^{S). 
2. Iffi rtl /2 then 

(A X /2)-^(Am) := {{u,v) G Pi X P2\fi{u) = f2{v)} 

is a smooth manifold with corners. Moreover if Pi are clean so is {fi x /2 ) ^ ^ ( Am ) ■ 

Suppose P is an oriented clean smooth manifold with comers. An orientation of P 
induces an orientation on diP and therefore an orientation on each component of diP. Let 
ai and a2 be two components of diP and (3 a component of 92 -P- Suppose that ^ C ai 
and /? C a2- Clearly /3 is a codimension one submanifold of the diP. Then an orientation 
o on P induces an orientation on ai which in tum induces an orientation on /3. 
Similarly the orientation on P induces an orientation o^^ on a2 which in tum induces an 
orientation o| on /3. Observe that the orientations ojj and | are opposite. 

Suppose now that P is a compact orientable clean smooth manifold with comers of 
dimension n. 

Fix an orientation on each component a of the comers. Each component a of dkP is 
a smooth manifold of dimension n — k. Denote by Vk the set of components of dimension 
k (the components of dn-kP)- 

Let Ik - Vk X Vk-i — >^ Z be the function defined by 



M} (/i X /2 rti Am). 



4(a,/3) 



< 



+1 ifP C a 
-lif^Ca 



and Oa induces 0^3 

and Oa induces the oposite of op 



aePfc,/3ePfc_i. 

The above observation implies that 




(8) 



for any a' e Vk+i, a" e Vk-i- 

If for a commutative ring k one considers the k— module 



C'' :=Maps(Pfe,K) 
and the hnear maps : ^ (7*^+^ defined by 



d\f){oc)= ^ 4+i(a,/3)/(/3) 
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f ^ , then the equality ([Sjl implies that (C*^, d*^) is a cochain complex. 

The cohomology of this cochain complex is called the incidence coliomology of the 
manifold with corners P. This cohomology is independent on the chosen orientations oj^s. 

4. Proof of the main theorem 



We will prove Theorem 1.4 only for W~ and T{x,y). The statements for will 
follow by changing X into —X since the stable sets for X are the unstable sets for —X. 
The statements for A4{x,y) can be verified in essentially the same way as for T(a;, j/). 
Alternatively they can be derived from the statements about W^T and in view of the 
fact that M{x,y) — x i+)~^(AM) consists of the pairs of points in the product 
W~ X equalized by and i+ . 

We will focus the attention to the assertions (1.) and (2.). Part of the assertion (3.), the 
orientability and the existence of the stable framing follow from the simple observation 
that T{x, y) X W~ is an open set of diW~ . The last part, the compatibility of the stable 
framings and of the orientability of T{x, y) for various x, y is a tedious but conceptually 
straightforward verification. More details will be provided in the expanded version of this 
work, cf. [5 1, which treats the more general case of Bott - hyperbolic vector fields. 

We first prove assertions (1.) and (2.) under the additional hypothesis H. 

Hypotliesis H: X admits a proper Lyapunov function / : M M which in the neigh- 
borhood of rest points, in convenient coordinates ui, • • • , it^, wi, • • • , Vn-k, is given by 
the quadratic expression 

k n—k 

2 



■ 

■1=1 ] = 1 

We also suppose that with respect to these coordinates the unstable resp. the stable set of 
the rest points corresponds to M'^ x resp. x M"^'''. 

Let • • • ci < C2 < ■ ■ ■ Ck < Ck+i • • • be the set of critical values. Choose e,; such that 
Ci+i - e^+i > Ci + Ci. 
Introduce 

and denote by the set of rest points which lie in Mi, X{i) — X D Mi. Denote by 

ifk : M- ^ Af+_i 

the map defined by the flow of X. Precisely (pk{x) is the intersection of the trajectory 
-fx, X e M~ with M+_j, see FIGURE 2. below. 
For X g X{i) denote by 



see FIGURE 3. below. 



u 

Df:= U 
(S+x §-),;:= y S+xS- 
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Figure 2. 



(S+>.D-), := y S+xD-. 
xex(i) 

Introduce the sets 

Pi — {{u, v) E X I u, V lie on the same possibly broken trajectory}, 
Qi — {{u, v) G X M{i) I u, V lie on the same possibly broken trajectory}. 



M(k) 




Figure 3. 
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Each set Pi resp. Qi is a union of two disjoint subsets Pi and dPi resp. Qi and dQi, 

o 

Pi = {{u, v) £ Pi \ u,v lie on the same unbroken trajectory} 
dPi ~ {{u, v) Pi \ u,v lie on the same broken trajectory} 

o 

Qi = {(""j v) £ Pi \ u,v lie on the same unbroken trajectory} 
dQi = {{u, v) £ Qi \ u,v lie on the same broken trajectory}. 
Equipped with the topology induced from the product x M~ resp. x M{i) 

Q O 

Pi resp. Qt is a topological manifold with boundary with Pj resp. Qi the interior and 

o o 

dPi resp. dQi the boundary. Actually both Pi and dPi resp. Qi and dQi are smooth 
submanifolds of x M~ resp. AI^ x M(i) but Pi resp. might not be in general. 

For both Pj and Qi denote by pj' resp. the projection on the first resp. second 
component of x and M^^ x M(i) and by Pi ~ p^ x p^ their product. The map 
Pi is one to one and homeomorphism onto the image. In addition we have the following. 

(a) The map p, identifies dP, with (S+ x S-)i, and dQ^ with (§+ x D"),. 

O 

(b) The restrictions of pf to Pj are diffeomorphisms onto M='= and the restrictions 
to dPi , via the identification above, are the projections on Sf . 

o 

(c) The restriction of to Qi is a smooth bundle over \ Sf with fiber an open 
interval and the restriction to dQi is the projection on S^. 

o 

(d) The restriction of p^ to Qi is a diffeomorphism onto M{i) \ and the restriction 
to dQi is the projection on D~. 

As pointed out above the subsets Pi{Pi) and Pi{Qi) are not necessarily smooth submani- 
folds of X M~ resp. x M{i), however the following propositions will provide 
structures of smooth manifold with boundary on both Pj and Qi which will make Pi smooth 
maps. 

Proposition 4.1. The smooth map pi : dPi —> x M-~ admits smooth extensions 

Pi : dP, X [0, e) M+ x Mr so that: 

1. the image is a neighborhood of {^'^ x inpi{Pi), 

2. Pi is injective, 

3. Pi restricted to dPi x (0, e) is of maximal rank. 

Proposition 4.2. The smooth map Pi : dQi — )• M^' x M{i) admits smooth extensions 

Pi : dQi X [0, e) M+ x M{i) so that: 

1. the image is a neighborhood of (S"*" x D" )i in pi (Qi) 

2. Pi is injective. 

3. Pi restricted to dQi x (0, e) is of maximal rank. 

The proofs of these propositions will be given towards the end of the section. 

Equip Pj with the smooth structure defined by the atlas obtained from {dPi x [0, e),pj} 

o 

and Pj. Similarly equip Qi with the smooth structure defined by the atlas obtained from 

o 

{dQi x [0, e),Pi} and Qi. Equivalently, regard Pj resp. Qi obtained by glueing dPi x [0, e) 

o o 

resp. dQi x [0, e) to P, resp. Qi via the diffeomorphisms provided by the restriction of 
Pi to dPi X (0, e) resp. to dQi x (0, e). These smooth structures wiU be denoted by {Pi)h 
resp.{Qi)h. 

If the rest points of X are of Morse type then we have the following. 
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Proposition 4.3. If the rest points of X are of Morse type then the image Pi(Pi) C x 
M~ resp. Pi{Qi) C x M(i) are smooth submanifolds with boundaries. 

The proof of this proposition will be given towards the end of the section. 

This implies that Pi resp. Qi have a smooth structure of manifold with boundary de- 
noted by {Pi)m resp. {Qi)m- The structures (m) are [h) are never the same but id : 
{Pi)h {Pi)m and id : {Qi)h ^ {Qi)m are smooth homeomorphisms which restrict to 
diffeomorphisms on the interiors and on the boundaries. 



Propositions 4.1 4.2 and 4.3 imply that for any (r, k) the product V := Ax P^+fc-i x 
Pr+k-2 ■ ■ ■ Pk+1 X B with A a smooth manifold and B a smooth manifold, possibly with 
boundary, is a smooth manifold with corners. The corner 9/7' can be described as follows. 

For any i with r + k— l>i>k + l denote by Ri the subset of Pi which is either the 
interior or the boundary of Pi and by Rk the subset of B which is either the interior or the 
boundary of B. 

Then the corner diV is the disjoint union of products A x R^+k-i x Rr+k-2 ■ ■ ■ Rk+i x 
Rk with I of the sets R[s being boundaries and the remaining r — I being interiors. For 
example if B is a smooth manifold with boundary and / = 1 



diV ^A X dPr+k-l X Pr+k-2 X • • • Pfc+1 X BU 
o o o 

A X Pr+k-1 X dPr+k-2 X • • • Pk+1 • • • X S U • • • 
o o o 

A X Pr+k-l X Pr+k-2 X • • • dPk+1 X BU 
o o o 

A X Pr+k-l X Pr+k-2 X • • • Pk+1 X dB . 



Suppose Propositions |4. 1 1 [4^2 4.3 were established. Here is the general scheme to verify 
(1.) and (2.) for f{x, y) and W- . 

For any r,k > consider the diagram 

Xr^k Xr+k-1 Xr+k-2 • • • 

.Vr+k-2 




FIGURE 4. 
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where X's, Y's and A are smooth manifolds, Z's and B smooth manifolds with boundary 
(possibly empty) and the arrows are smooth maps with ip's embeddings. 
Denote by S,0 and V the spaces defined by: 

(a) S = S{r + k, k) X^+k x Xr+k-i x • • • Xk+i, 

(b) O = 0{r + fc, k) := Xr+k x Yr+k-i x Xr+k-i x • • • Xk+i x Yfc, 

(c) V = V{r + k, k) := A X Zr+k-i x Zr+k-2 X • • • Zfc+i X B, 
Denote by s and t the maps defined by: 

(a) s = s{r + k,k) : S ^ O, the product of all maps from the top line to the middle 
hne in the diagram above and 

(b) t = t{r + k,k) : P ^ O, the product of all maps from the bottom to the middle 
line in diagram above. 

Since id and ip's are embeddings so is s and s{S) is a smooth submanifold of O. Note that 
T-" is a smooth manifold with comers, O and S are smooth manifolds, t and s are smooth 
maps. 



We say that " the diagram FIGURE 4. is transversal" if i rh s{S). If so by Theorem 3.3 
the subspace t^^{s{S)) receives a structure of smooth manifold with corners. 

To accomplish the proof of (1.) and (2.) in Theorem 1.4 we choose the integers r, k, the 
manifolds X's, ■ ■ ■ , B and the maps (pi,pf , a, /3, appropriately in order to obtain T(x, y) 
and {ix)^^{f^^{cr+k,Cr)), as t^^{s{S)). Then we verify the transversality of the diagram 
FIGURE 4. 

The case of'T{x, y). Choose (r, k) so that f{x) = Cr+k and f{y) = Ck- 
TakeA = S'-,B = S'+, 

{ip,+i : X,+i ^ Yi) : Mr^^ ^ M+), k<i<r + k-l, 

iZj,p+,pj) ^{P^,p+,p-), k + l<3 <r + k-l. 

Take a, /3 to be the obvious inclusions. With these choices f^^{s{S)) identifies with 
f{x,y). 

Diagram FIGURE 4 becomes 




FIGURES. 
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Then (1.) and (2.) follow from Pi'oposition |4.4| below. 
Proposition 4.4. The diagram (FIGURE 5) is transversal. 



Propositions 4.4 is a consequence of the transversality rh i+ for y G X{X). For 
details the reader can consult ^ and ||3]- 

The case ofW^T : Let f{x) = c,,,. and k < m. This case is treated in two steps. 

First we check that the open set {ix)^^{M{k)) has a structure of a smooth manifold 
with corners. For this purpose we use the same diagram (FIGURE 4) for {r = m — k, k) 
X, y, Z, A, a as in the case of Tix, y), B — Qk, 13 — : Qk ^ Mjt and we replace 



Proposition |4.4| by Pi'oposition |4.5| below. Diagram FIGURE 4 becomes 

^r+k-l 
id 




FIGURE 6. 

Proposition 4.5. The diagram (FIGURE 6) is transversal. 



As with Proposition 



4.4 



Proposition 4.5 is a consequences of the transversality rh iy 



for x,y € X( X). F or details the reader can consult |2| and |3| 



4.5 



implies that [ix) {M{k)) has a structure of smooth manifold with 



Proposition 
corners. 

Second, we verify that the smooth structures on {ix)^^{M{k)) and on {irc)^^{M{k — 
1)) agree. For this purpose we consider the map h := f o : Qi ^ (ci+i, Ci_i) and let 
Q'^ := h~^{ci+i, Ci) and Q" := h~^{ci, Cj_i). Both are open subsets of Qi and we have: 

Observation 4.6. There are canonical diffeomorphisms 

e^Q'fe^ A/+ X {ck+i,ck) 

where the fiber product is taken with respect to p^ : Pk ^ M^^ and the projection x 

(cfe,Cfc-i) MjT. 

Then the composition of : 

(a) ei : Ql ^ Pk x^,- {M^ x (cfc,Cfe_i)), 
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(b) the inclusion Pfe x^^- (M^. x {ck,Ck-i)) C Pk x [M,, x {ck.cu-i)), 

k 

(c) id X (pk-i X id : Pfe X Af^ x (cfe,Cfe_i) Pk X- M,t_i x (cfe,Cfe_i) and 

(d) id X : Pfe X M+_^ X (cfe,Cfe._i) ^ Pfe X g',._i 

is a smooth embedding denoted by 6k : QJ,' Pk 'x Q'k-i- 

We write t' : 7^' ^ O resp. t" : V" ^ O instead of t : P ^ O and Q' resp. Q" 
instead of Q. 



In view of Observation 4.6 the map 7^'(n, A:) — V'^n, k — 1), given by the product of 
id's (on , Pn-i, ■ ■ ■ Pk-i) and of 9k : Q'l PkX Q'^^^, is a smooth embedding which 
sends 

{t"{n,k)r\s{n,k){Sin,k)) 



onto 

{t'{n, k - l))-\s{n, k - l){S{n, k - 1)). 

It identifies the structures of smooth manifolds with corners which were derived using k 
and k — 1. 

Apparently the smooth structures defined so far depend on the Lyapunov function and 
the choices of ; this is not the case. 

The independence of Lyapunov function: The arguments are the same for T and W~ 
so we will treat only T. 

If / and /' are two Lyapunov functions and 7 a possibly broken instanton we consider 
two transversals V = {Vi, ■ ■ ■ , 14) and V' = {V-[, ■ ■ ■ V^) with the same mark points 
and Vi contained in the levels of / and V^ contained in the levels of /'. We can find 
a diffeomorphism of a neighborhood [/ of 7 onto a neighborhood 9{U) of 7 which 
restricts to the identity on 7 and sends Vi into V^ . This diffeomorphism provides an open 
embedding from the product of Vi into the product of V[ . It follows that id : T — > T is 
smooth and of maximal rank in the neighborhood of 7 with respect to either one of the 
smooth structure (h) or (m) defined using / and /'. 

The removal of the additional hypothesis H: While global Lyapunov functions might 
not exist, for any broken instanton 7 from the rest point x to the rest point y one can find 
an open neighborhood U in M so that a "convenient Lyapunov function" /:[/—> M for 
X\(j exists. Here "convenient Lyapunov function" means that the system {X\i;, f,U) is 
diffeomorphic to (y |y , t/jy , where y is a smooth vector field satisfying Pi and P2 on a 
smooth manifold N, g : N ^ Ra proper Lyapunov function for Y and V an open set in A^. 
As the space 7(7 (a;, y) consisting of broken instantons from xtoy which lie in U is an open 
set in T(a;, y) we define a smooth structure on Tu{x, y) and note that for different such 
U' s these structures agree on intersections. The smooth structure on Tu{x, y) is defined 
using the space of broken instantons of F on which lie in V. 

Proof of Proposition \4.1\ \4.2\ 

First we introduce some notation. In the context of Theorem |2.1| in section 2 denote by 
and by the sphere and the disc of radius e in and when this notation is applied 
to coordinates about a rest point x write and by instead. 
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X D- X [0,oo) (I 
1^), by the formulae: 



) and X2 



Define the maps xi = iXi^Xi ) ■ 
ixt,X2)-S+ xD- x[0,^)^{R+ X] 

Xi{p,q,s) =7"(-l/s,p,g, -l/s, l/s)i/s=^0 (9) 

xJCp,?,-?) =7+(1/s,p,(7, -1/s,1/s) i/s 7^ 

X^(p,g,0)=0 (10) 

X2"(p,'7,s) =9. 

Define x ■= {xi,X2) : S+ x D- x [0, 00) (M+ x M") x (M+ x R-). 
Clearly x(5'+ x S*- x [0,oo)) C (5+ x R-) x (M+ x §-). 

The estimates in Theorem |2. 1 [ show that the map x is smooth and for 9 small the restric- 
tion of X to 5"+ X 5^ X (0, 9) and to 5*+ x S*^ x is of maximal rank but x is not. It fails 
at the points of x x 0. 

Example : If 

k n 

X = — grad J — — ^ Xid/dx 

i=l j=k+l 

Xk), q = {xk+1, ■ ■■ ,Xn) 



Xjd/dxj 



(11) 



a simple calculation shows that: 

i~^{t;p,q,Ti,T2) 



q- 



The estimates in Theorem 2.1 are satisfied and x is visibly not of maximal rank at the 
points of 5*+ X 5^ X 0. 



We proceed now with the proof of Propositions 4.1 and 4.2 



Observe that it suffices to check the statements in Propositions |4.1| and |4.2| for small 
enough and the statement in Proposition 4.2 for M{i) replaced by the smaller open set 

/"Hcz - ei,Ci + ei). 

Choose for each rest point x e a ne ighborhood and coordinates in the neighbor- 
hood so that the hypotheses of Theorem 2.1 are satisfied and f — Ci — l/2|pp + l/2|gp. 
Here \p\ and \q\ denote the norm in the respective coordinates. Choose = e/2 with e 



small enough to have the conclusions of Theorem |2.1 [ satisfied for each rest point. 

Since there is no risk of confusion from now on we drop the index i from notation and 
write M{e) instead of f^^{ci — e, Ci + e). 

Define -u+ : 5+ x — > M+,resp. : D+ x S*^ — > to be the map which 
assigns to {p, q) the intersection of the trajectory through {p, q) with A/+ resp. . The 
maps u+ and u~ are diffeomorphisms on their images and their restrictions to 5+ x resp. 
to X are the identity maps. 

For the rest point x denote by (xf , X2) maps (xi, X2) defined by the formulae ^ 
and ([T0|. 

For P take p^ {u+ o xf , o xf ) : 5*+ x 5" x [0, e') ^ M+ x M" and for Q take 
Pj. :— (u+ o X17X2) ■ ^ ^ [0,^) — >■ X M{e) with 9 small enough to insure 
that the image of x x [0, 9) by xf li^s in M(e). Take p = Li^^x(i)Px- The maps p 
satisfy the conclusions of Propositions 4.1 and|4.2| 
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Proof of Proposition \4. 3\ We use the same conventions and notations as in the previous 
proof. 

The chosen neighborhoods and coordinates for the rest points are so made to have X 



given by ( 1 1 1. Then, each trajectory of X passing through (p, q) e M" x M" ^ ( k = ind x) 
at s is given by 

7(s) = (7"^(s),7~(s)), 7+(s)=e"'*p, 7"(s)=e'*g. 

To check that P is a smooth submanifold with boundary it suffices to construct the 
smooth maps a;^; : 5*+ x S"^ x [0, e) M+ x so that u : (§+ x x [0, e) 
M+ X M~ defined by w = U^-g x{i)^x satisfies: 

la. (jj restricted to dP = (§+ x is the identity, 

2a. the image of a; is an open neighborhood of dP in P. 

3a. u! is of maximal rank on (S+ x x [0, e), (Note the distinction between item 3a. 
above and item 3. in Propositions |4.1| and |4.2| ) 

Define — {u^,u^) by assigning to {p,q,t), p e ,q £ and t e [0, e) the pair 
of points provided by the intersection of the trajectory passing through p + tq, with A/+ 
and Define Wj. := (u+ o x f,u^ ° xl) where 

x''iP,(l,t) ■■= {Xi{p,q,t),X2iP^q,t) := (j),tq,tp,q). 

Items la., 2a, 3a. above are satisfied. 

To check that Q is a smooth submanifold with boundary it suffices to construct the 
smooth maps u)^ : x x [0, e) Af + x M{e) so that w — ^x£X{i)^x satisfies: 

lb. Lo restricted to dQ — (§+ x ) is the identity, 

2b. the image of lo is an open neighborhood of dQ in Q, 

3b. LO is of maximal rank on (§+ x D^) x [0, e). 

Define 7i+ by assigning to (p, q,t), p G S^,q E the intersection of the trajectory 
passing through tp + q with . Define lu^ {u^ o xf, xl) where 

X{u,v,t) = iXi{p,q,t),X2iP^<l,'t)) {(P^tq), {tp,q)). 

Items lb, 2b, 3b. above are satisfied, 
q.e.d 

Observation 4.7. The reader can notice that the smooth structures (h) and (m) in the case 
of a vector field with the rest points of Morse type provided by Propositions |4. 1[ |4.2| and 
Propos ition |4 . 3 | re spectively can not be the same. 
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